The present invention relates to an electronic watermark insertion device suitable for digital images, and particularly to an electronic watermark insertion device for inserting electronic watermark into digital images.
Recently, illegal copies of digital images have become an important problem. Unlike analog images, since digital images are recognized by 0s and 1s, they can be repeatedly replicated without any degradation in image quality. This inherent feature has caused great damage to digital image copyright holders.
In order to prevent such illegal replication, it is considered to prepare a reproduction system that encrypts digital image data and has a valid secret decryption key whereby encrypted digital image data can be reproduced. However, after the encryption is once decoded, this system cannot prevent subsequent replication.
In order to prevent digital images from being illegally used or replicated, the method has been considered of burying special information (hereinafter referred to as xe2x80x9celectronic watermark dataxe2x80x9d) in a digital image itself.
Two types of data including visible electronic watermark data and invisible electronic watermark data are considered as electronic watermark data for digital images.
The visible electronic watermark data, which contains special characters or symbols combined with an image, can be visually sensed. This electronic watermark data may degrade the image quality but has the advantage of visually warning users to prevent misappropriation of digital images.
An example of burying such visible electronic watermark data is disclosed in JP-A-241403/1996. This patent publication discloses a method of placing a visible watermark on a digital image. This method consists of the steps of supplying an original digital image, supplying a digital watermark image, and superimposing the watermark on the original image, without changing the chromaticity of pixels of the original image upon watermarking, to create a watermarked image. This method is characterized in that the image-creating step includes the step of varying the brightness (not color) of respective opaque pixels in a watermarked image to correct pixels corresponding to the original image. In this method, only the brightness of pixels corresponding to opaque portions of electronic watermark data is varied so that visible electronic watermark data is synthesized with the original image without changing the color components. The scaling value of varying the pixel brightness component depends on color components, random numbers, pixel values of electronic watermark data, or others. The prominence of the watermark is determined by the scaling value.
In invisible electronic watermark data, electronic watermark data is buried in an image, in consideration of degradation of image quality. Since the image quality degradation is not substantially negligible, the watermark cannot be visually recognized.
As described above, since special information, which can be recognized by an author, is buried as the electronic watermark data, the author can be specified by detecting the electronic watermark data even after illegal replication. Moreover, information about replication disapproval may be buried in an image. In such a case, when the reproduction unit, for example, detects the replication disapproval information, the reproduction by a VTR or the equivalent can be restricted by informing the user that the detected information is reproduction prohibited data, or by operating the replication preventing mechanism within the reproduction unit.
As one method of burying invisible electronic watermark data into digital images, special information is buried as electronic watermark data in portions not substantially affecting the image quality, such as LSBs (least significant bits) of pixel data. However, according to this method, the electronic watermark data can be easily removed from images. For example, information regarding LSBs of pixels will be missed using a low-pass filter. The image compression process discards the volume of information not adversely affecting the image quality, thus reducing the volume of data. This means that the electronic watermark data is lost. As a result, the problem is that it is difficult to re-detect the electronic watermark data.
JP-A-No. 315131/1994 shows another example of the electronic watermark burying method. This publication discloses the method of burying specific information by using the correlation between continuous frame images and detecting the area where degradation in image quality does not occur even when substitution is performed in peripheral areas upon reproduction. According to this method, an image is reconstituted by specifying an identification data buried area using the signal dropout portion and conversion information upon reproduction and then by correcting the corresponding portion.
As further another example, JP-A-No. 30466/1993 discloses the method of converting the frequency of a video signal and then burying information with signals of frequencies lower than the frequency band of the converted video signal. In this method, a broad band-pass filter extracts the original video signal while a low-pass filter extracts the buried identification data.
In another example, the method of frequency-converting images and then burying electronic watermark data into portions with strong frequency components of a video signal after the frequency conversion (see xe2x80x9cNIKKEI Electronicsxe2x80x9d, 1996, 4.22 (no. 660), page 13). In this method, since electronic watermark data is buried into frequency components, the electronic watermark data is not lost through the compression process or filtering image process. Moreover, using the random numbers with a normal distribution as electronic watermark data makes it difficult to prevent interference between electronic watermark data and to destroy the electronic watermark data without significantly affecting the entire image.
In the electronic watermark data burying method, the original image is first transformed into frequency components by the DCT (discrete cosine transformation) 703. n pieces of data with high values over high frequency range are selected as f(l), f(2), . . . , f(n). The electronic watermark data, w(l), w(2), . . . w(n), are selected from a normal distribution having an average of 0 and a dispersion of 1. The formula, F(i)=f(i)+xcex1xc3x97|f(i)|xc3x97w(i), where xcex1 is a scaling factor, is calculated to obtain respective (i)s. Finally, the image in which the electronic watermark data is buried is obtained based on the frequency component in which f(i) is substituted for F(i).
Moreover, the electronic watermark data is detected according to the following method. In this detection method, both the original image and electronic watermark data candidate w(i) (where i=1, 2, . . . , n) must be known.
First, the image containing electronic watermark data is converted into frequency components through, for example, DCT. Values corresponding to factor values, f(1), f(2), . . . , f(n), each containing an electronic watermark, are set as F(1), F(2), . . . , F(n), respectively. The formula, W(i)=(F(i)xe2x88x92f(i))/f(i), is solved using f(i) and F(i) to extract the electronic watermark data W(i). Next, the statistical similarity C between w(i) and W(i) is obtained by the following formula including a vector inner product.
C=Wxe2x80xa2w/(WDxc3x97wD)
where W=(W(1), W(2), . . . , W(n); w=(w(1), w(2), . . . , w(n)); WD=the absolute value of a vector W; wD=the absolute value of a vector w; and the symbol xe2x80xa2 represents an inner product.
When the statistical similarity C is more than a specific value, it is judged that the electronic watermark data is in a buried state.
The above-mentioned method, where the electronic watermark data is buried into an image, is effective when an author holding an original image detects digital image data suspected as an illegal replicate.
In the above-mentioned method that requires an original image, the author, that is, an original image owner, can detect image data doubted as an illegal replicate. However, the reproduction unit in each terminal cannot detect electronic watermark data because of the absence of the original image.
To overcome that problem, an improvement of the above-mentioned method for the terminal processing has been proposed. In the improved method, the original image is divided into blocks each having K pixelsxc3x97K pixels. Electronic watermark data is buried or extracted in block process units.
In the electronic watermark data burying process, AC frequency components are set as f(1), f(2), . . . , f(n) in a frequency increasing order over a frequency range after the discrete cosine transformation. The electronic watermark data, w(l), w(2), . . . , w(n) are selected from the normal distribution having an average of 0 and a dispersion of 1. In order to obtain respective (i)s, the formula of F(i)=f(i)+xcex1xc3x97avg(f(i))xc3x97w(i) is calculated, where xcex1 is a scaling factor and avg(f(i)) is a partial average obtained by averaging the absolute values at three points adjacent to f(i). An image in which electronic watermark data is buried can be obtained from the frequency components where f(i) is substituted with F(i).
Electronic watermark data is detected according to the following method. This method does not require any original image. It is merely required that electronic watermark data candidate w(i) (where i=1, 2, . . . , n) is known.
Over the frequency band where electronic watermark data contained image is subjected to a discrete cosine transformation, the frequency components are set as F(1), F(2), . . . , F(n) in a frequency increasing order. The average of the absolute values of three adjacent points in the watermark data F(i) is set to a partial average avg(F(i)). The electronic watermark data W(i) is obtained by calculating the following formula.
W(i)=F(i)/avg(F(i))
Moreover, the sum WF(i) is obtained by calculating w(i) for each frame according to the above-mentioned formula. Next the statistical similarity C between w(i) and WF(i) is obtained by calculating the following formula including an vector inner product:
C=WFxe2x80xa2w/(WFDxc3x97wD)
where WF=(WF(1), WF(2), . . . , WF(n)); w=(w(1), w(2), . . . , w(n)); WFD=the absolute value of a vector WF; wD=the absolute value of a vector w; and the symbol xe2x80xa2 represents an inner product. When the statistical similarity C is more than a specific value, it is judged that the corresponding electronic watermark data is in a buried state.
FIG. 3 shows a prior art electronic watermark inserting device employing the above-mentioned method. Referring to FIG. 3, the input image 310 is transformed from the time region to the frequency region by means of the discrete cosine transformation unit 320. The electronic watermark burying unit 320 inserts the electronic watermark data 340 in the resultant data. The buried electronic watermark data is transformed from the frequency region to the time region by means of the inverse discrete cosine transformation unit 350. Thus, the output image 360 with the inserted electronic watermark data can be obtained.
However, the prior-art device may insert evenly and repeatedly electronic watermark data even in an image with the same.
FIG. 4 schematically shows the concept of burying electronic watermark data in a frequency region. The frequency spectrum 440 after the electronic watermark data burying operation is formed by adding the frequency spectrum 420 of electronic watermark data to the frequency region of an input image. Allocating a specific region of an input image does not depend on the type of electronic watermark data. When the electronic watermark data burying process is performed plural times, the same electronic watermark data is again buried on the frequency spectrum of an image with data previously buried. If the same electronic watermark data is buried twice or three times, the upper black portion corresponding to the buried amount becomes larger in the frequency spectrum 440xe2x80x2 after the burying process. When the frequency region of the spectrum is converted into the time region, the burying noises become larger, so that images are emphasized with the noise components.
Each piece of electronic watermark data is inserted as very feeble noises, compared with the original image and is not visually recognized by a human eye. However, there is the problem in that if noises repeatedly integrated are large, it may be sensed by the human eye, thus leading to degradation in image quality.
The present invention is made to solve the above-mentioned problems.
Moreover, the objective of the invention is to provide an electronic watermark data burying device that can bury the same electronic watermark data into an image only once, not duplicatively.
The objective of the present invention is achieved by an electronic watermark insertion device comprising discrete cosine transforming means for subjecting an input image to a discrete cosine transformation; electronic watermark burying means for inserting electronic watermark data into data transformed by the discrete cosine transforming means; electronic watermark detecting means for detecting whether or not the electronic watermark data to be inserted by the electronic watermark burying means has been already inserted into the input image; and selecting means for selecting the output data from the discrete cosine transforming means when the electronic watermark detecting means detects that the electronic watermark data has been already inserted into the input image or for selecting output data from the electronic watermark burying means when the electronic watermark detecting means does not detect that the electronic watermark data has been inserted into the input image; and means for subjecting data selected by the selecting means to an inverse discrete cosine transformation.
In the electronic watermark insertion device according to the present invention, the electronic watermark detecting means decides that the corresponding electronic watermark data is buried, when a statistical similarity C is more than a specific value.
Moreover, in the electronic watermark insertion device according to the present invention, the electronic watermark detecting means first orthogonally converts an image with the electronic watermark data into a frequency component; calculates and extracts electronic watermark data W(i) using the formula W(i)=(F(i)xe2x88x92f(i))/f(i), where both an original image and electronic watermark data candidate wi (where i=1, 2, . . . n) are known; where f(1), f(2), . . . f(n) to which electronic watermarks are respectively buried in a frequency range correspond factor values F(1), F(2), . . . , F(n); next calculates a statistical similarity C between w(i) and W(i) using the formula, C=Wxe2x80xa2w/(WDxc3x97wD), where W=(W(1), W(2), . . . W(n)), w=(w(1), w(2), . . . w(n)), WD=the absolute value of a vector W, and wD=the absolute value of a vector w), and the symbol xe2x80xa2 represents a vector inner product; and detects that the electronic watermark data is in a buried state when the statistical similarity C is more than a specific value.
In the electronic watermark insertion device according to the present invention, the electronic watermark burying means sets AC frequency components (f(1), f(2), . . . , f(n)) in a frequency increasing order over a frequency range after a discrete cosine transformation; selects electronic watermark data, w(1), w(2), . . . , w(n), from normal distribution having an average of 0 and a dispersion of 1; and calculates respective (i)s the formula, F(i)=f(i)+xcex1xc3x97|f(i)|xc3x97w(i) (where xcex1 is a scaling factor) so that an image, in which the electronic watermark data is buried, is obtained from the frequency component in which f(i) is substituted with F(i).
Moreover, in the electronic watermark insertion device according to the present invention, the electronic watermark burying means sets AC frequency components (f(l), f(2), . . . , f(n)) in a frequency increasing order over a frequency range after a discrete cosine transformation; selects electronic watermark data, w(1), w(2), . . . , w(n), from normal distribution having an average of 0 and a dispersion of 1; and calculates respective (i)s using the formula, F(i)=f(i)+xcex1xc3x97avg|f(i)|xc3x97w(i) (where xcex1 is a scaling factor and avg f(i) is a partial average of absolute values of f(i) at three adjacent points) so that an image, in which the electronic watermark data is buried, is obtained from the frequency component in which f(i) is substituted with F(i).
Moreover, according to the present invention, an electronic watermark insertion device comprises orthogonal converting means for subjecting an input image to an orthogonal conversion; electronic watermark burying means for inserting electronic watermark data to data converted by the orthogonal converting means; electronic watermark detecting means for detecting whether or not the electronic watermark data to be inserted by the electronic watermark burying means has been already inserted into the input image; selecting means for selecting the output data from the orthogonal converting means when the electronic watermark detecting means detects the electronic watermark data already inserted into the input image or for selecting output data from the electronic watermark burying means when the electronic watermark detecting means does not detect the electronic watermark data inserted into the input image; and means for subjecting data selected by the selecting means into an inverse discrete cosine transformation.
Furthermore, according to the present invention, an electronic watermark insertion device comprises orthogonal converting means for subjecting an input image to an orthogonal conversion; electronic watermark detecting means for detecting whether or not electronic watermark data to be inserted has been already inserted into the input image; electronic watermark data selecting means for selecting electronic watermark data by which the input image is not affected in data insertion when the electronic watermark detecting means detects the electronic watermark data already inserted into the input image or for selecting the electronic watermark data to be inserted when the electronic watermark detecting means does not detect the electronic watermark data inserted into the input image; electronic watermark burying means for inserting electric watermark data selected by the electronic watermark data selecting means into data converted by the orthogonal converting means; and means for subjecting data into which the electronic watermark data is buried by the electronic watermark burying means, to an inverse orthogonal conversion.
In the electronic watermark insertion device according to the present invention, the electronic watermark detecting means decides that the corresponding electronic watermark data is buried, when a statistical similarity C is more than a specific value.
In the electronic watermark inserting device according to the present invention, the orthogonal converting means performs discrete Fourier transformation, discrete cosine transformation, Hadamard-Walsh transformation, or Karhunen-Loeve transformation.